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1.  ABSTRACT 


The  objective  of  this  project  is  to  develop  theoretical  foundations,  based  upon  the  broad 
framework  of  the  theory  of  fuzzy  (or  nonadditive)  measures,  and  methodology,  supported 
by  appropriate  computer  software,  for  systems  modeling  with  imprecise  probabilities  of 
various  kinds.  This  objective  is  motivated  by  the  recognition  that  the  assumption  that 
relevant  probabilities  can  be  assessed  precisely  is  highly  unrealistic.  Results  obtained  by 
the  work  dedicated  to  achieving  this  objective  are  summarized  in  this  report;  their 
complete  presentation  is  covered  in  43  project-related  publications,  including  two 
doctoral  dissertations,  which  are  listed  in  Sec.  10  of  this  report  (pp.31-34).  The  results 
are  loosely  classified  into  the  following  seven  areas:  1.  Complete  methodology  for 
Bayesian  inference  based  upon  interval- valued  or  fuzzy  probabilities  and  likelihoods;  2. 
Well-justified  information  measure  in  Dempster-Shafer  theory;  3.  Relevant  theoretical 
results  in  fuzzy  measure  theory,  a  theory  that  is  connected  with  imprecise  probabilities  in 
a  similar  way  as  classical  measure  theory  is  connected  with  classical  (precise) 
probabilities;  4.  Procedures  for  constmcting  imprecise  probabilities  and  fuzzy  measures 
by  various  methods,  including  the  use  of  neural  networks  and  genetic  algorithms;  5. 
Various  other  theoretical  results  that  emerged  from  the  work  on  the  project,  including  (i) 
Complete  representations  of  Dempster-Shafer  theory  and  possibility  theory  in  terms  of  the 
usual  semantics  of  propositional  modal  logic,  (ii)  Basic  ideas  of  mathematics  of  finite 
resolution,  (Hi)  Constrained  fuzzy  arithmetic,  (iv)  A  method  for  identifying  key  variables 
in  systems  modeling  via  fuzzy  c-means  clustering  based  on  varying  distance  function,  and 
(v)  A  thorough  mathematical  analysis  of  the  well-known  Cox’s  proof,  by  which  it  is 
shown  that  the  proof  does  not  demonstrate  the  inevitability  of  the  rules  of  classical 
Bayesian  inference,  as  often  claimed;  6.  Applications  of  the  Bayesian  inference  with 
fuzzy  probabilities  to  the  problem  of  military  unit  identification  and  to  statistical  decision 
making;  and  7.  Computer  programs  for  some  algorithms  that  emerged  from  the  work  on 
this  project. 


2.  INTRODUCTION 

It  has  increasingly  been  recognized  that  practical  applicability  of  classical  probability 
theory  to  some  problems  of  current  interest,  particularly  those  dependent  on  human 
judgment,  is  severely  restricted.  This  restriction  is  a  result  of  the  additivity  axiom  of 
classical  probability  theory.  Due  to  this  axiom,  elementary  events  are  required  to  be 
pairwise  disjoint  and  the  probability  of  each  is  required  to  be  expressed  precisely  by  a  real 
number  in  the  unit  interval  [0,1].  The  requirement  of  disjoint  events  makes 
mathematically  good  sense,  but  it  becomes  problematic  whenever  we  leave  the  world  of 
mathematics.  As  is  well  known,  there  is  no  method  of  scientific  measurement  that  is  free 
from  error.  As  a  consequence,  observations  in  close  neighborhoods  of  the  sharp 
boundaries  between  events  are  unreliable  and  should  be  properly  discounted.  This  results 
in  a  violation  of  the  additivity  axioms  and,  by  implication,  in  a  violation  of  the  precision 
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requirement.  Hence,  we  need  to  deal  with  imprecise  probabilities.  This  need  is  even 
more  pronounced  when  instead  of  measurements  we  are  dependent  on  assessments  based 
on  subjective  human  Judgment.  There  are  other  convincing  arguments  for  imprecise 
probabilities,  including,  for  example  the  following: 

•  Imprecision  of  probabilities  is  needed  to  reflect  the  amount  of  information  on  which 
they  are  based.  The  precision  should  increase  with  the  amount  of  information 
available. 

•  Total  ignorance  can  be  properly  modeled  by  vacuous  probabilities,  which  are 
maximally  imprecise  (i.e.,  each  covers  the  whole  range  [0,1]),  but  not  by  any  precise 
probabilities. 

•  Imprecise  probabilities  are  generally  easier  to  assess  and  elicit  than  precise  ones. 

•  We  may  be  unable  to  assess  probabilities  precisely  in  practice,  even  if  that  is  possible 
in  principle,  because  we  lack  the  time  and  computational  ability. 

•  A  precise  probability  model  that  is  defined  on  some  class  of  events  determines  only 
imprecise  probabilities  for  events  outside  the  class. 

•  When  several  sources  of  information  (sensors,  experts,  and  individuals  in  a  group 
decision)  are  combined,  the  extent  to  which  they  are  consistent  can  be  reflected  in  the 
precision  of  the  combined  model. 

The  imprecision  in  expressing  probabilities  introduces  a  new  dimension  into  statistical 
modeling.  It  can  be  utilized,  for  example,  to  reflect  the  amount  of  information  upon 
which  probabilities  are  estimated  or  the  degree  of  conflict  in  information  obtained  from 
several  different  sources.  These  issues  are  crucial  in  statistical  modeling  with  small 
observation  sets  as  well  as  in  problems  of  information  fusion. 

The  first  thorough  study  of  imprecise  probabilities  was  undertaken  by  Walley  [1991].  His 
principal  result  is  a  demonstration  that  reasoning  and  decision  making  based  on  imprecise 
probabilities  satisfies  the  principles  of  coherence  and  avoidance  of  sure  loss,  which  are 
generally  viewed  as  principles  of  rationality.  Hence,  the  requirement  of  precision  (or, 
equivalently,  the  additivity  axiom)  cannot  be  justified  as  inevitable  for  rationality,  as 
previously  believed.  The  soundness  of  this  project  is  based  on  this  important  result. 

In  general,  a  probability  model  avoids  sure  loss  if  its  use  cannot  lead  to  outcomes  that  are 
harmful  to  the  user.  It  is  coherent  if  it  does  not  contain  any  internal  inconsistencies 
resulting  from  the  lack  of  transitivity  of  a  relevant  preference  ordering.  These  concepts 
are  carefully  formalized  by  Walley  [1991],  and  he  shows  that  coherent  models  can  be 
constructed  from  any  set  of  probability  assessments  that  avoid  sure  loss  by  a 
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mathematical  procedure  called  a  natural  extension.  Based  on  the  perceived  needs  for 
dealing  with  imprecise  probabilities,  the  objective  of  this  project  is  to  develop  theoretical 
foundations  and  methodology  for  systems  modeling  with  imprecise  probabilities.  It  was 
realized  that  imprecise  probabilities  of  different  types  must  be  distinguished  since  they 
require  different  methodological  treatments.  The  imprecision  of  probabilities  can  be 
expressed,  for  example,  by  intervals  of  real  numbers,  convex  sets  of  probability 
distributions,  fuzzy  numbers,  or  fuzzy  intervals.  In  this  project,  these  various  types  of 
imprecise  probabilities  are  treated  as  special  cases  within  fuzzy  measure  theory,  a 
relatively  new  theory  concerned  with  nonadditive  measures,  whose  connection  with 
imprecise  probabilities  is  analogous  to  the  connection  of  classical  measure  theory  with 
precise  probabilities. 

The  purpose  of  this  report  is  to  present  a  summary  of  results  obtained  by  working  on  this 
project.  It  is  organized  as  follows.  An  overview  of  basic  issues  addressed  and  type  of 
results  obtained  in  this  project  is  presented  in  Sec.  3.  Reviewing  relevant  background 
knowledge  in  Sec  4  follows  this.  The  principal  parts  of  the  report  are  Sections.  5-8,  in 
which  the  actual  results  obtained  in  the  project  are  summarized  at  the  conceptual  level. 
These  sections  contain  frequent  references  to  43  project-related  publications,  including 
two  doctoral  dissertations,  where  the  results  are  presented  in  all  technical  details.  The 
project-related  publications  are  listed  chronologically  in  Sec.  10,  and  numbers  in  square 
brackets  makes  references  to  them.  Other  references  are  listed  in  Sec.  9,  and  author’s 
name  and  year  of  publication  make  references  to  them. 

In  all  papers  associated  with  this  project,  terminology  and  notation  regarding  fuzzy 
measures  is  adopted  from  [Wang  and  Klir  1992].  Terminology  and  notation  regarding 
other  areas  is  adopted  from  [Klir  and  Yuan,  1995]. 

Conclusions  summarizing  the  significance  of  the  obtained  results  are  presented  in  Sec.  8, 
together  with  a  discussion  about  prospective  future  directions  of  research  pertaining  to 
imprecise  probabilities  and  related  areas. 

The  work  on  the  project  was  headed  through  the  whole  period  by  two  faculty  co¬ 
investigators,  George  J.  Klir  (Principal  Investigator)  and  Zhenyuan  Wang.  Nine  research 
assistants,  Jiri  Fridrich,  David  Harmanec,  Yin  Pan,  Sylvia  Rabeler,  Jaron  Rubenstein, 
John  F.  Swan-Stone,  Jia  Wang,  Wei  Wang,  and  Bo  Yuan,  worked  on  various  aspects  of 
the  project  and  to  various  degrees. 


4 


3.  PROJECT  OVERVIEW 


This  project  is  based  on  the  recognition  that  fuzzy  measure  theory  relates  to  imprecise 
probabilities  in  a  similar  way  as  classical  measure  theory  [Halmos  1950]  relates  to  precise 
probabilities.  Based  on  this  recognition,  it  was  decided  to  utilize  fuzzy  measure  theory  as 
a  general  framework  within  which  imprecise  probabilities  can  be  investigated  in  rigorous 
and  comprehensive  way.  Although  the  theory  was  fairly  well  developed  when  this  project 
was  initiated,  research  was  still  needed  to  make  it  fully  operational  for  its  intended  use  in 
this  project. 


Results  obtained  by  working  on  this  project  are  loosely  classified  into  the  following  seven 
areas: 

1.  Complete  methodology  was  developed  for  Bayesian  inference  based  on  imprecise 
probabilities,  in  which  prior  probabilities  and  likelihoods  are  either  interval-valued  or 
fuzzy  [10,25,33,38].  This  methodology  includes  a  novel  procedure  for  constructing  fuzzy 
prior  probabilities  from  opinions  of  selected  experts.  This  procedures  is  based  on  pairwise 
comparisons  of  given  alternatives  according  to  each  of  several  given  criteria,  as  well  as 
pairwise  comparisons  of  the  criteria  [24,33]. 

2.  Efficient  algorithms  were  developed  [17]  for  computing  theoretically  well  justified 
measures  of  uncertainty  and  uncertainty-based  information  established  by  Harmanec  and 
Klir  [1994]  in  two  theories  of  uncertainty.  Both  theories  are  distinct  from  classical 
probability  theory  and  may  be  viewed  as  special  theories  of  imprecise  probabilities: 
Dempster-Shafer  theory  of  evidence  [Shafer  1976,  Yager  et  al.  1994]  and  possibility 
theory  [Dubois  and  Prade  1988,  De  Cooman  1997].  Based  on  these  uncertainty  measures, 
specific  algorithms  were  also  developed  for  information-preserving  transformations  [5] 
from  Dempster-Shafer  theory  to  either  possibility  theory  or  probability  theory 
(approximations),  and  between  possibility  theory  and  probability  theory  (in  both 
directions)  [3,  18,  30].  The  new  results  are  integrated  with  relevant  results  obtained 
earlier  by  the  principal  investigator  in  three  invited  technical  papers  [19,29,31],  a 
technical  monograph  [32],  and  an  expository  paper  regarding  the  changing  attitudes 
toward  uncertainty  and  imprecision  (including  imprecise  probabilities)  in  science  [8]. 
Moreover,  a  well-justified  measure  of  nonspecificity  (one  of  several  recognized  types  of 
uncertainty)  was  established  for  systems  defined  on  the  Euclidean  space  of  any  finite 
number  of  dimensions  [6]. 

3.  Numerous  theoretical  results  were  obtained  in  fuzzy  measure  theory  in  the  process  of 
utilizing  the  theory  for  investigating  certain  fundamental  issues  of  imprecise  probabilities, 
such  as  the  issue  of  natural  extensions  of  lower  probabilities.  Some  of  these  results 
[1,9,26,37,42]  directly  contribute  to  the  general  theory  of  imprecise  probabilities,  as 
conceived  by  Walley  [1991].  Other  results  in  this  category  [14,15,16,27]  are  more 
fundamental  to  fuzzy  measure  theory  itself,  but  were  needed  as  prerequisites  for  pursuing 
research  leading  eventually  to  the  results  more  directly  connected  with  this  project. 
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4.  The  important  issue  of  constructing  imprecise  probabilities  or  associated  fuzzy 
measures  under  various  scenarios  was  thoroughly  investigated.  Several  distinct 
procedures  emerged  from  this  investigation  [4,11,12,13,23,39,40].  The  use  of  neural 
networks  [23,40]  and  genetic  algorithms  [11,39]  was  found  particularly  fitting  for  this 
kind  of  problems. 

5.  Various  other  results  emerged  from  the  project.  These  results  were  obtained  by 
broadening  various  special  problems  regarding  imprecise  probabilities.  Each  of  these 
results  opens  a  new  direction  for  future  research.  They  include:  (/)  Complete 
representations  of  Dempster-Shafer  theory  and  possibility  theory  in  terms  of  the  usual 
semantics  of  propositional  modal  logic  [3];  (ii)  Basic  ideas  of  mathematics  of  finite 
resolution,  which  is  a  general  formalism  for  dealing  with  imprecision  in  all  forms  and, 
hence,  is  potentially  relevant  for  dealing  with  imprecise  probabilities  [20,21];  (Hi) 
Principles  of  constrained  fuzzy  arithmetic,  which  are  needed  for  operating  on  fuzzy 
probabilities  (e.g.,  in  the  Bayesian  inference  based  on  fuzzy  probabilities)  and  are  also 
essential  in  many  other  contexts  [22,34,35];  (iv)  A  method  for  identifying  key  variables  in 
data-driven  systems  modeling  via  fuzzy  c-means  clustering  algorithm  and  an  evolutionary 
program  by  which  the  distance  function  employed  in  the  clustering  is  appropriately 
modified  [7,23,36];  and  (v)  A  thorough  mathematical  analysis  of  the  well-known  Cox’s 
proof,  by  which  it  is  shown  that  the  proof  does  not  demonstrate  the  inevitability  of  the 
rules  of  classical  Bayesian  inference,  as  often  claimed  [2,41]. 

6.  Two  applications  of  the  Bayesian  inference  with  interval-valued  or  fuzzy  prior 
probabilities  and  likelihoods  were  developed  [33].  The  first  application  deals  with  the 
problem  of  military  unit  identification.  While  Bayesian  inference  is  a  proper  tool  for  this 
problem,  the  requirement  of  classical  Bayesian  inference  that  all  probabilities  be  precise 
is  not  realistic  due  to  high  observation  uncertainties,  which  are  in  this  case  unavoidable. 
The  second  application  deals  with  statistical  decision  making  in  face  of  imprecise  prior 
probabilities  and  likelihoods. 

7.  Computer  programs  were  developed  to  support  some  methodological  tools  resulting 
from  the  project.  These  are  C+-t-  programs  implementing  the  following  algorithms: 

A.  Four  versions  of  Bayesian  inference  with  imprecise  probabilities  [10,25,33,38].  The 
meaning  of  prior  probabilities  and  likelihoods  in  each  of  these  versions  is  shown  in  Table 
1.  It  is  clear  that  any  combination  that  is  not  specifically  handled  by  these  versions, 
including  precise  prior  probabilities,  can  be  handled  as  a  special  case  by  the  version 
closest  to  it. 
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Likelihoods 

Prior  probabilities 

Intervals 

Precise 

Version  1 

Version  3 

Intervals 

Version  2 

Version  4 

Table  1:  Meaning  of  the  individual  versions  of  programs  developed  for 
Bayesian  inference  with  imprecise  probabilities. 

B.  Computation  of  uncertainty  measures  in  Dempster-Shafer  theory  [17].  Two  versions 
were  proposed:  one  suitable  (computationally  efficient)  for  bodies  of  evidence  with  small 
numbers  of  focal  elements;  the  other  suitable  to  those  with  large  numbers  of  focal 
elements. 

C.  Computation  of  natural  extensions  with  respect  to  given  lower  probabilities  [37].  The 
concept  of  natural  extension  in  the  theory  of  imprecise  probabilities  is  a  generalization  of 
the  concept  of  expectation  in  classical  probability  theory. 

D.  Constructing  ?i-fuzzy  measures  [Sugeno  1974]  from  input-output  data  by  a  special 
genetic  algorithm  [11,39].  ^-measures  are  convenient  for  representing  interval-valued 
probabilities. 

Further  information  about  these  programs  is  covered  in  Appendix  of  this  report. 


4.  BACKGROUND 

4.1  Review  of  Fuzzy  (Nonadditive)  Measures 

Given  a  universal  set  X  and  a  non-empty  family  C  of  subsets  of  X  (usually  with  an 
appropriate  algebraic  structure),  a  fuzzy  measure  (or  nonadditive  measure),  /i,  on  (X,^  is 
a  function 

IJL :  e  ^[0,°o] 

that  satisfies  the  following  requirements: 

(^1)  pi  (0)=O  (vanishing  at  the  empty  set); 

(ji2)  for  all  A,B  e  if  AeB,  then  pi  (A)<  pi  (B)  (monotonicity); 

{pi3)  for  any  increasing  sequence  AiC  A2. .  .of  sets  in 

if  U^^A.  e  then  .l^^(Ap  = 

{continuity  from  below)'. 
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(fl4)  for  any  decreasing  sequence  Ai  3  A2  ...  of  sets  in 
if  n^jA.  e  then  .lim  ^i(Ap  = 

{continuity  from  above). 

Functions  that  satisfy  requirements  (fil),  (fil),  and  either  (/x3)  or  (^4)  are  equally 
important  in  fuzzy  measure  theory.  These  functions  are  called  semicontinuous  from 
below  or  from  above,  respectively.  When  the  universal  set  X  is  finite,  requirements  (jj3) 
and  (IJ.4)  are  trivially  satisfied  and  may  thus  be  disregarded. 

When  fuzzy  measures  are  utilized  for  characterizing  uncertainty  (e.g.,  lower  or  upper 
probabilities),  their  domain  S  is  usually  a  a-algebra  of  subsets  of  X  (often  the  full  power 
set,  P(X),  of  X),  their  range  is  the  unit  interval  [0,1],  and  it  is  required  that  //(X)=l. 

For  any  pair  A,  B  g  ^  such  that  A  n  B  =  0,  a  fuzzy  measure  /t  is  capable  of  capturing 
any  of  the  following  situations  with  respect  to  the  measured  property: 

(a)  /r  (AuB)  >  /t  (A)  +  fi  (B)  {super additivity),  which  expresses  a 
cooperative  action  or  synergy  between  A  and  B; 

(b)  p  (AuB)  =  p  (A)  +  p  (B)  {additivity),  which  expresses  the  fact  that  A 
and  B  are  noninteractive; 

(c)  p  (AuB)  <  p  (A)  +  p  (B)  {subadditivity),  which  expresses  some  sort  of 
incompatibility  between  A  and  B. 

Observe  that  classical  probability  theory,  which  is  based  on  classical  measure  theory 
[Halmos  1950],  is  capable  of  capturing  only  situation  (b).  This  demonstrates  that  fuzzy 
measure  theory  provides  us  with  a  considerably  broader  framework  than  probability 
theory  for  formalizing  uncertainty.  As  a  consequence,  it  allows  us  to  capture  types  of 
uncertainty  that  are  beyond  the  scope  of  probability  theory. 

A  special  class  of  fuzzy  measures  that  can  be  utilized  for  expressing  imprecise 
probabilities  are  measures  that  satisfy  the  following  special  axiom:  for  all  A,  B  e  if  A 

n  B  =  0,  then 


p  {AkjB)=p  (A)  +p{B)  +  Xp  {A)p  (B) 

where  (-I,®®)  is  a  parameter  by  which  different  measures  in  this  class  are  distinguished. 
These  measures,  which  are  easy  to  handle,  are  usually  called  X-fuzzy  measures  or  Sugeno 
measures  [Sugeno  1974].  These  measures  are  superadditive  when  A->0,  additive  when 
^=0,  and  subadditive  when  A.<0. 
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4.2  Review  of  Dempster-Shafer  Theory  (DST) 

DST  is  based  on  two  dual  nonadditive  measures:  belief  measures  and  plausibility 
measures.  Given  a  universal  set  X  (usually  referred  to  as  the  frame  of  discernment  in 
DST),  assumed  here  to  be  finite,  a  belief  measure  is  a  function 

Bel :  P(X)  ^  [0,1] 

such  that  Bel(0)  =  0,  Bel(X)  =  1,  and 

Bel(A,  uA2U---uA„)>XBel(Aj)-XBel(Aj  nAJ 

j  j<k 

+---+(-l)"^'Bel(A,nA2n---nA„)  (1) 

for  all  possible  families  of  subsets  of  X.  Due  to  the  inequality  (1),  belief  measures  are 
superadditive.  When  X  is  infinite,  function  Bel  is  also  required  to  be  continuous  from 
above.  A  plausibility  measure  is  a  function 

PI :  P(X)  [0,1] 


such  that  P1(0)  =  0,  P1(X)  =  1,  and 

P1(A,  n  A^n- •  -nAJ  <  XpKAj)  -  XpKAj  u  A,)  (2) 

j  j<k 

+  •  •  •  +  (-1)"+*  Pl(Ai  u  A2  u  •  •  •  u  An) 

for  all  possible  families  of  subsets  of  X.  Due  to  the  inequality  (2),  plausibility  measures 
are  subadditive.  When  X  is  infinite,  function  PI  is  also  required  to  be  continuous  from 
below. 

It  is  well  known  that  either  of  the  two  measures  is  uniquely  determined  from  the  other 
one  by  the  equation 


P1(A)  =  1  -  Bel(A)  (3) 

for  all  A  e  P(X).  It  is  also  well  known  that 

P1(A)  >  Bel(A)  (4) 

for  each  A  e  ^(X).  In  the  special  case  of  equality  in  (4)  for  all  A  e  P(X),  we  obtain  a 
classical  (additive)  probability  measure. 
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DST,  which  has  become  an  important  tool  for  dealing  with  uncertainty  [Yager,  et  al. 
1994],  is  best  covered  in  a  book  by  Shafer  [1976].  Its  position  in  fuzzy  measure  theory  is 
described  by  Wang  and  Klir  [38]. 

Belief  and  plausibility  measures  can  conveniently  be  characterized  by  a  function 

m:  P(X) [0,1], 

which  is  required  to  satisfy  two  conditions: 

(i)  m(0)  =  O; 

(ii)  Xi“(A)  =  l- 

AeP(X) 

This  function  is  called  a  basic  probability  assignment.  For  each  set  A  e  P(X),  the  value 
m(A)  expresses  the  proportion  to  which  all  available  and  relevant  evidence  supports  the 
claim  that  a  particular  element  of  X,  whose  characterization  in  terms  of  relevant  attributes 
is  deficient,  belongs  to  the  set  A.  This  value,  m(A),  pertains  solely  to  one  set,  set  A;  it 
does  not  imply  any  additional  claims  regarding  subsets  of  A.  If  there  is  some  additional 
evidence  supporting  the  claim  that  the  element  belongs  to  a  subset  of  A,  say  B  c  A,  it 
must  be  expressed  by  another  value  m(B). 

Given  a  basic  probability  assignment  m,  the  corresponding  belief  measure  and  plausibility 
measure  are  determined  for  all  sets  A  e  P(X)  by  the  formulas 

Bel(A)  =  X  (5) 

b|bca 

P1(A)=  Xm(B).  (6) 

B|Ar^B#0 

Inverse  procedure  is  also  possible.  Given,  for  example,  a  belief  measure  Bel,  the 
corresponding  basic  probability  assignment  m  is  determined  for  all  A  e  P(X)  by  the 
formula 


m(A)=  X(-l)'^""'Bel(B),  (7) 

b|bca 

as  proven  by  Shafer  [1976].  Hence,  each  of  the  three  functions,  m,  Bel  and  PI,  is 
sufficient  to  determine  the  other  two. 

Given  a  basic  probability  assignment,  every  set  A  g  P(X)  for  which  m(A)  5^  0  is  called  a 
focal  element.  The  pair  (?,  m),  where  ^  denotes  the  set  of  all  focal  elements  induced  by 
m  is  called  a  body  of  evidence. 
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Total  ignorance  is  expressed  in  evidence  theory  by  m(X)  =  1  and  m(A)  =  0  for  all  A^X. 
Full  certainty  is  expressed  by  m({x})  =  1  for  one  particular  element  of  X  and  m(A)  =  0 
for  all  A  ^  {x}. 

To  investigate  ways  of  measuring  the  amount  of  uncertainty  represented  by  each  body  of 
evidence  (Sec.  6.2),  it  is  essential  to  understand  properties  of  bodies  of  evidence  whose 
focal  elements  are  subsets  of  the  Cartesian  product  of  two  sets.  That  is,  we  need  to 
examine  basic  probability  assignments  of  the  form 

m:P(XxY)^[0,l], 


where  X  and  Y  denote  universal  sets  pertaining  to  two  distinct  domains  of  inquiry  (e.g., 
two  investigated  variables),  which  may  be  connected  in  some  fashion.  Let  m  of  this  form 
be  called  a  joint  basic  probability  assignment. 

In  this  case,  each  focal  element  induced  by  m  is  a  binary  relation  R  on  X  x  Y.  When  R  is 
projected  on  set  X  and  on  set  Y,  we  obtain,  respectively,  the  sets 


and 


Rx  =  {x  €  XI  (x,y)  e  R  for  some  y  e  Y} 


Ry  =  {y  e  Y  I  (x,y)  e  R  for  some  x  e  X}. 

These  sets  are  instrumental  in  calculating  marginal  basic  probability  assignments  mx  and 
my  from  the  given  Joint  assignment  m: 


=  X  all  A  e  P{X), 

RIA  =  Rx 

my  (B)  =  X  for  B  e  P(Y). 

RIB=Ry 

Let  bodies  of  evidence  associated  with  mx  and  my,  (?x.  nax)  and  (?y,  my),  be  ealled 
marginal  bodies  of  evidence.  These  bodies  are  said  to  be  noninteractive  if  and  only  if  for 
all  A  e  7x  and  all  B  e  ?y 


m(A  X  B)  =  mx(A)  •  my  (B)  (8) 

and 

m(R)  =  0  for  all  R  A  X  B. 

It  is  well  known  [Shafer  1976]  that  the  belief  measure  and  its  dual  plausibility  measure 
eollapse  into  a  single  measure  whenever  all  focal  elements  are  singletons.  This  measure 
is  additive  and,  henee,  it  is  a  probability  measure. 

Any  probability  measure.  Pro,  on  a  finite  set  X  can  be  uniquely  determined  by  a 
probability  distribution  function 
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p:X->[0,l] 


via  the  formula 


Pro(A)  =  Xp(x). 

xeA 


From  the  standpoint  of  evidence  theory,  clearly 

p(x)  =  m({x}) 

for  all  X  s  X. 

Probability  measures  are  special  belief  measures  (as  well  as  plausibility  measures)  and, 
hence,  are  subject  to  the  same  rules  as  the  latter.  The  various  special  features  of 
probability  theory  are  well  known  and  are  not  covered  here. 

It  is  well  known  [Wang  and  Klir  1992]  that  X,-fuzzy  measures  introduced  in  Sec.4.1  are 
closely  associated  with  belief  and  plausibility  measures  of  DST.  Each  A,-fuzzy  measure  is 
also  a  belief  measure  when  ^  >  0,  a  plausibility  measure  when  <  0,  and  a  probability 
measure  when  X  =  0.  Although  Vfuzzy  measures  do  not  capture  all  belief  and  plausibility 
measures,  they  are  considerably  easier  to  represent  and  manipulate.  Hence,  they  are  often 
used  instead  of  DST  to  reduce  computational  complexity. 


4.3  Review  of  Possibility  Theory 

Possibility  theory  is  based  on  two  dual  semicontinuous  fuzzy  measures,  called  possibility 
measures  and  necessity  measures.  A  possibility  measure,  Pos,  is  characterized  by  the 
property 


Pos(Ui^,Ai)  =  supPos(Ai) 

i€l 


(9) 


for  any  family  of  subsets  of  X  defined  by  an  arbitrary  index  set  I,  where  sup  denotes  the 
supremum  (least  upper  bound).  All  possibility  measures  are  semicontinuous  from  below. 
A  necessity  measure,  Nec,  is  characterized  by  the  property 

^®^*^niei^i)  =  ^”fNec(Ai)  (10) 

for  any  family  of  subsets  of  X,  where  inf  denotes  the  infimum  (greatest  lower  bound).  All 
necessity  measures  are  semicontinuous  from  above. 

Given  a  possibility  measure  Pos,  a  function  r  on  X  defined  by  the  equation 
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r(x)  =  Pos({x}) 


(11) 


for  all  X  G  X  is  called  a  possibility  distribution  function.  This  function  uniquely 
characterizes  the  possibility  measure  via  the  formula 

Pos(A)  =  supr(x)  (12) 

xeA 


for  all  A  e  P(X).  Moreover,  given  a  possibility  measure  Pos,  there  exists  a  necessity 
measure,  Nec,  that  is  dual  with  respect  to  Pos  in  the  sense  that 

Nec(A)  =  1  -  Pos(A)  (13) 

for  all  A  G  P(X).  Several  other  important  properties  of  possibility  and  necessity 
measures  can  readily  be  derived  [Dubois  and  Prade  1988]: 


Pos  (A  n  B)  <  min[Pos(A),  Pos(B)], 

(14) 

Nec(A  u  B)  >  max[Nee(A),  Nec(B)], 

(15) 

Pos(A)  <  1  ^  Nec(A)  =  0 

(16) 

Nec(A)  >  0  =>  Pos(A)  =  1 

(17) 

t  us  now  assume  that  X  is  a  finite  set. 

Let  X  =  [xi,  X2, 

for  all  A,  B  G  P(X). 


Xn}  and  let  q  denote  r  (Xi)  for  all  i  =  1,2,  n.  Assume  further  (without  any  loss  of 
generality)  that  q  >  q+i  for  all  i  =  1,2,  ...,  n  -  1.  Then  the  basic  probability  assignment 
function  m  associated  with  r  is  given  by  the  formula 


m(A)  = 


[q  -q,,,  for  A  =  Ei,  i  =  l,2,---,n 
lo  for  A Ej  for  all  i  =  l,2,-”,n 


(18) 


for  all  A  G  P(X),  where  Ei  =  {xi,  xi,  Xijfor  all  i  =  1,2,  ...,  n  and  rn+i  =  0  by 
convention.  The  inverse  formula  is 

r-SmCE,)  (19) 

k=i 

for  all  i  =  1,2,  ...,  n  [Klir  and  Yuan  1995], 

Possibility  theory  may  be  viewed  as  a  special  branch  of  DST,  in  which  focal  elements  are 
always  nested  [Shafer  1976].  This  makes  possibility  theory  naturally  suited  for 
representing  evidenee  in  terms  of  fuzzy  sets,  since  a-cuts  of  fuzzy  sets  are  also  nested 
(See.  4.4).  To  explain  this  connection,  let  “X  denote  a  variable  that  takes  values  in  a 
universal  set  X,  and  let  information  about  the  actual  value  of  the  variable  be  expressed  by 
a  fuzzy  proposition  is  F,”  where  F  is  a  fuzzy  set  on  X.  To  express  this  information  in 
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measure-theoretic  terms,  it  is  natural  to  interpret  the  membership  degree  F(x)  for  each  x  e 
X  as  the  degree  of  possibility  that  =  x.  This  interpretation  induces  a  possibility 
distribution  function,  rp,  on  X  that  is  defined  by  the  equation 

rplx)  =  F(x)+ 1  -hp  (20) 


for  all  X  e  X,  where 


hp  =  supF(x) 

xeX 


Eq.  (20)  applies  to  both  normal  fuzzy  sets  (hF=l)  and  subnormal  fuzzy  sets  (hp<l )  [Klir 
1998], 


4.4  Review  of  Fuzzy  Probabilities 

To  define  fuzzy  probabilities,  we  need  to  explain  first  the  concept  of  a  fuzzy  set.  In 
general,  boundaries  of  fuzzy  sets  are  not  required  to  be  precise.  Each  fuzzy  set  is  fully 
characterized  by  a  function  that  assigns  to  each  element  of  a  designated  universal  set  its 
degree  of  membership  in  the  fuzzy  set,  usually  in  the  unit  interval  [0,1].  This  function  is 
called  a  membership  function.  A  fuzzy  set  A  is  thus  usually  expressed  in  terms  of  a 
membership  function  of  the  form 


A:  X->[0,1], 

where  X  is  a  crisp  (non  fuzzy)  universal  set  under  consideration.  For  each  xeX,  the  value 
A(x)  designates  the  degree  of  membership  of  x  in  A  or,  alternatively,  the  degree  of 
compatibility  of  X  with  the  concept  represented  by  A. 

The  largest  membership  degree  in  a  given  fuzzy  set  A  is  called  its  height  and  is  denoted 
by  hA.  More  precisely. 


h^  =  supA(x), 

X€X 


(21) 


where  sup  denotes  the  supremum.  When  hA=l,  A  is  called  a  normal  fuzzy  set,  otherwise, 
it  is  called  subnormal.  When  A  is  defined  on  a  finite  universal  set  X,  the  number 

|A|  =  Xa(x)  (22) 

xeX 


is  called  a  scalar  cardinality  of  A  or,  by  some  authors,  a  sigma  count  of  A. 
For  every  ae  [0,1],  a  given  fuzzy  set  A  yields  the  crisp  set 
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“A  =  {xgXI  A(x)  >  a}. 


(23) 


which  is  called  an  a-cut.  Since  ai  <  a2  implies  A2“^  A ,  the  family  of  all  distinct  a- 
cuts  of  any  fuzzy  set  forms  a  nested  sequence  of  crisp  sets.  This  sequence  uniquely 
represents  the  given  fuzzy  set  via  the  formula 

A(x)  =  sup  a  •  “A(x)  (24) 

ae[0,l] 


for  all  xeX  [Klir  and  Yuan  1995].  This  connection  between  fuzzy  sets  and  crisp  sets 
makes  it  possible  to  generalize  properties  of  crisp  sets  into  their  fuzzy  counterparts.  A 
property  of  crisp  sets  is  generalized  into  fuzzy  sets  when  it  is  preserved  (in  the  classical, 
crisp  sense)  in  all  a-cuts  of  relevant  fuzzy  sets.  Properties  of  classical  sets  that  are 
generalized  into  fuzzy  sets  in  this  way  are  called  cutworthy.  It  is  important  to  realize  that 
only  some  properties  of  fuzzy  sets  are  cutworthy. 

Fuzzy  sets  that  are  defined  on  the  set  of  real  numbers,  R,  are  called  fuzzy  quantities. 
Some  fuzzy  quantities  are  particularly  useful  since  they  can  be  employed  as  meaningful 
approximators  of  closed  intervals  of  real  numbers  or  individual  real  numbers.  They  are 
called  fuzzy  intervals  and  fuzzy  numbers,  respectively.  A  fuzzy  interval  is  any  normal 
fuzzy  quantity  with  bounded  support  whose  a-cuts  for  all  as  (0,1]  are  closed  intervals  of 
real  numbers  [Klir  and  Yuan,  1995}.  A  fuzzy  number  A  is  a  special  fuzzy  interval  for 
which  A(x)=l  for  exactly  one  xe  r. 

Each  fuzzy  interval  may  conveniently  be  expressed  for  all  xe  R  in  the  canonical  form 


A(x)  = 


fA(x) 

when  X  e  [a,b) 

1 

when  X  e  [b,c] 

gA(x) 

when  X  e  (c,d] 

0 

otherwise. 

(25) 


where  a,  b,  c,  d  e  R  are  such  that  a^<c<d,  fA  is  a  real-valued  function  that  is  increasing 
and  right-continuous,  and  gA  is  a  real-valued  function  that  is  decreasing  and  left- 
continuous. 

For  any  fuzzy  interval  A  expressed  in  the  canonical  form  of  Eq.  (25),  the  a-cuts  of  A  are 
expressed  for  all  ae  (0,1]  by  the  formula 

a^^|[f;'(a),gA(a)]  when  a  (26) 

[[b,c]  when  a  =  1, 
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where  and  g'^  are  the  inverse  functions  of  4  and  ,  respectively . 


When  functions  fA  and  gA  in  (25)  are  linear  functions,  A  is  called  a  trapezoidal  fuzzy 
interval.  This  type  is  often  employed  in  applications  since  it  is  easy  to  represent  and 
manipulate.  Each  trapezoidal  fuzzy  interval  is  fully  characterized  by  the  quadruple  (a,  b, 
c,  d)  of  real  numbers  in  (25)  via  the  special  canonical  form 


fx-  a 


A(x)  = 


b  -  a 
1 

d  -  X 


d  -c 
0 


when  xG[a,b) 
when  XG[b,c] 
whenxG(c,d] 
otherwise 


(27) 


It  is  convenient  to  use  A=  (a,  b,  c,  d)  as  a  shorthand  symbol  representing  this  special 
form.  When  b=c,  we  obtain  a  triangular  fuzzy  number. 

It  follows  immediately  form  Eq.  (26)  and  Eq.  (27)  that  a-cuts  of  trapezoidal  fuzzy 
intervals  are  expressed  for  all  aG  (0,1]  by  the  formula 

“  A  =  [a  +  (b  -  a)a,  d  -  (d  -  c)a].  (28) 

Fuzzy  intervals  and  fuzzy  numbers  are  important  for  this  project  since  each  fuzzy 
probability  is  a  fuzzy  interval  or  a  fuzzy  number  defined  on  the  unit  interval  [0,1].  In 
general,  we  can  apply  fuzzy  arithmetic  on  fuzzy  intervals  (or  numbers)  and,  hence,  on 
fuzzy  probabilities  [Kaufman  and  Gupta  1985,  Klir  and  Yuan  1995].  However,  since 
probabilities  must  add  to  one  and  due  to  some  other  reasons  explained  in  Sec.  7.2, 
standard  fuzzy  arithmetic  is  not  applicable  and  must  be  replaced  with  constrained  fuzzy 
arithmetic  [22,34,35]. 


5.  BAYESIAN  INFERENCE  WITH  IMPRECISE  PROBABILITIES 
5.1  Interval-Valued  Probabilities 

Interval-valued  probabilities  are  perhaps  the  most  common  imprecise  probabilities.  They 
are  easy  to  represent  and  manipulate,  and  have  a  meaningful  and  useful  interpretation  in 
many  applications.  In  addition,  they  are  essential  for  formulating  and  dealing  with  fuzzy 
probabilities,  as  explained  in  Sec.  5.2. 

Results  regarding  Bayesian  inference  with  interval-valued  probabilities,  which  are  only 
summarized  in  this  section,  are  presented  in  detail  in  Refs.  [33,  38]. 
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Let  S  be  a  finite  sample  space  with  a  probability  measure  p.,  and  let  be  a  discrete 
random  variable  with  values  in  the  set  X={xi  I  i  GNn},  where  Nn={  1,2,  ...,n}.  Subsets  of 
S  are  called  events.  For  any  Ac  S,  p(A)  denotes  the  probability  of  event  A.  For  each 
XiG  X,  there  is  a  unique  event 


Ai={seS  l;?:(s)  =  Xi}, 

in  terms  of  which  we  define  the  probability  of  ^l'=Xi,  denoted  by  p(Xi),  as 


Then,  the  n-tuple 


p(Xi)  =  p  (Ai). 


p  =  <p(xi)  I  ie  Nn> 

is  called  a  probability  distribution  on  X.  For  the  sake  of  simplicity,  let  ai  =  p(Xi)  for  all 
ieNn. 


Assume  now  that  each  of  the  probabilities  p(xi)=ai  in  p  is  not  known  exactly,  but  we 
know  its  lower  estimate,  a; ,  and  upper  estimate,  ai .  That  is,  we  only  know  that 

aj  e[aj,ai]. 

Then,  the  probability  distribution  p  becomes  a  tuple  of  probability  intervals  (TPI) 

I  =  ([ai,ai]  |ieN„) 

Each  TPI  represents  a  set  of  probability  distributions.  Pi,  defined  as 

Pj  =  j  (ajl  i  G  N„)  I  aj  G  [a^ai]  for  all  i  g  and  ^a; 

[  isN„ 

We  say  that  a  given  TPI  I  is  reasonable  if  its  associated  set  of  probability  distributions  Pi 
is  not  empty.  Whenever  Pi  is  empty,  this  indicates  that  the  estimates  employed  in  I  are 
conflicting  with  one  another  and,  hence,  they  are  totally  unusable.  In  such  a  case,  the 
assessments  must  be  properly  revised. 

An  important  subset  of  reasonable  TPI’s  are  called  feasible.  A  reasonable  TPI  I  is 
feasible  if  for  each  ieN^  and  every  z  g  [a^, a,] there  exists  at  least  one  probability 
distribution  in  Pi  such  as  ai  =  z. 

For  any  reasonable  TPI, 
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I  =  ([aj,ai]  I  i  eN„^, 

there  exists  a  unique  feasible  TPI 

such  that  Pj,  =  Pi  .  This  unique  TPI  is  obtained  for  all  ie  Nn  by  the  formulas 


a  :  =  max 


a  i  =  max 


a;,  l-X^j 

j^i  J 

ii,  i-X^j 

J 


(29) 

(30) 


Clearly,  [^j,a'i]c[aj,ai]  forallie  N„.  That  is,  among  all  reasonable  TPTs  that  are 

associated  with  the  same  set  of  probability  distributions,  the  unique  feasible  TPI  is 
defined  in  terms  of  the  smallest  intervals. 

In  the  developed  Bayesian  inference  based  on  interval-valued  probabilities  [38],  prior 
probabilities  are  always  interval-valued  while  likelihoods  are  either  precise  or  interval¬ 
valued. 

Applying  the  formula  of  Bayesian  inference  to  interval-valued  probabilities  means  that 
interval  arithmetic  must  be  used.  However,  the  standard  interval  arithmetic  [Moore 
1966]  is  not  directly  applicable  due  to  the  requirement  of  probability  theory  that 
probabilities  in  each  probability  distribution  add  to  1.  Hence  constrained  interval 
arithmetic  must  be  employed  (Sec.  7.2). 

Given  a  prior  TPI  that  is  feasible  and  likelihoods  (exact  or  interval-valued),  it  was  proven 
by  Pan  and  Klir  [38]  that  the  corresponding  posterior  TPI  can  always  be  determined  by 
two  linear  programming  problems.  These  problems  are  employed,  respectively,  for 
calculating  lower  and  upper  bounds  of  the  posterior  TPI.  It  was  also  proven  that  the 
posterior  TPI  obtained  by  solving  the  linear  programming  problems  are  always  feasible. 


Formulation  of  the  linear  programming  problems,  all  supporting  mathematical  proofs, 
and  various  related  algorithmic  considerations  are  covered  in  Refs.  [33,  38]. 
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5.2  Fuzzy  Probabilities 


Bayesian  inference  in  terms  of  fuzzy  probabilities,  as  developed  in  this  project  [10,  25, 
33],  is  a  generalization  of  results  obtained  for  Bayesian  inference  with  interval-valued 
probabilities.  To  connect  the  two  versions,  the  a-cut  representation  of  fuzzy  probabilities 
was  employed  (Sec.  4.4). 

Bayesian  inference  with  fuzzy  probabilities  deals  with  tuples  of  fuzzy  probabilities  (TFP), 
which  for  each  a  g  (0,1]  become  TPI’s 

“I  =  ([ai(a),ai(a)]  |  ieN„). 

That  is,  each  lower  bound  Uj  and  upper  bound  ai  in  a  TFP  is  a  function  of  a.  For  each 
i  G  ,  the  two  functions,  U;  (a)  and  a,  (a),  define  a  fuzzy  probability  by  Eq.(26)  via  the 
canonical  form  (25)  introduced  in  Sec.  4.4. 

The  computation  of  fuzzy  posterior  probabilities  is  based  on  similar  linear  programming 
problems  as  the  computation  of  interval-valued  posterior  probabilities.  However,  the 
former  linear  programming  problems  involve,  in  addition,  the  parameter  a  that  keeps 
track  of  the  meaning  of  each  TPI  in  terms  of  the  associated  TFP. 

For  the  sake  of  computational  efficiency,  fuzzy  prior  probabilities,  as  well  as  likelihoods 
are  assumed  to  be  trapezoidal  fuzzy  intervals  or  triangular  fuzzy  numbers.  Due  to  the 
nature  of  processing  of  the  fuzzy  probabilities,  these  shapes  are  usually  not  fully 
preserved  in  the  resulting  fuzzy  posterior  probabilities.  To  employ  the  posterior 
probabilities  obtained  in  a  particular  step  of  Bayesian  inference  as  prior  probabilities  in 
the  next  step,  it  is  desirable  (for  the  sake  of  computational  efficiency)  to  approximate 
them  by  trapezoidal  fuzzy  intervals  or  triangular  fuzzy  numbers.  A  simple  approximation, 
which  has  commonly  been  used  for  this  purpose,  is  obtained  by  keeping  the  values  a,  b,  c, 
d  in  (25)  unchanged  and  replacing  functions  fA  and  gA  in  (25)  with  their  linear 
counterparts 


The  accumulated  error  resulting  from  this  approximation  for  repeated  use  of  the 
multiplication  operation  was  examined  by  Giacheti  and  Young  [1998].  They  showed  that 
the  error  can  be  sufficiently  large  to  produce  misleading  results.  They  proposed  an 
alternative  approximation  and  showed  that  it  leads  to  acceptable  results  in  most  practical 
cases.  Moreover,  they  derived  an  error  expression  for  the  new  method,  which  can  be  used 
for  checking  whether  or  not  the  obtained  results  are  acceptable. 
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Another  approximation  was  proposed  by  Pan  [33].  It  is  based  on  keeping  only  values  b 
and  c  in  (25)  unchanged,  and  calculating  new  values  for  a  and  d  in  (25)  in  terms  of  a 
weighted  least-square-error;  the  weights  are  monotone  increasing  with  values  of  a 
according  to  some  rule.  Although  this  approximation  method  seems  promising,  no  error 
analysis  has  been  made  for  it  yet. 


5.3  Assessment  of  Prior  Probabilities 

An  important  issue  in  Bayesian  inference  is  the  determination  of  prior  probabilities, 
particularly  when  this  involves  human  judgment.  This  issue  was  addressed  by  Saaty 
[1980]  in  terms  of  his  well-known  method  of  hierarchical  analysis.  In  this  method, 
selected  experts  are  asked  to  make  pairwise  ratio  comparisons  of  given  alternatives  for 
each  of  several  relevant  criteria,  as  well  as  pairwise  ratio  comparisons  of  the  criteria 
themselves.  These  comparisons  are  then  appropriately  processed  to  obtain  overall 
pairwise  ratio  comparisons  of  the  alternatives  and  a  value  of  an  index  by  which  the  degree 
of  consistency  of  the  given  experts’  judgments  is  expressed. 

In  this  project,  a  counterpart  of  Saaty’ s  method  was  developed,  which  is  based  on 
differences  rather  than  ratios  [24,  33].  An  advantage  of  this  method  is  its  significantly 
lower  computational  complexity  since  all  operations  of  multiplication  and  division  in 
Saaty’ s  method  are  replaced,  respectively,  with  the  operations  of  addition  and  subtraction 
in  Pan’s  method.  The  method  was  also  generalized  to  allow  the  experts  to  express  their 
comparisons  in  terms  of  fuzzy  numbers.  This  produces  either  fuzzy  prior  probabilities  or, 
when  defuzzified,  precise  prior  probabilities. 


6.  DEMPSTER-SHAFER  THEORY  (DST) 


6.1  Construction  of  Belief  Measures 

Belief  measures  and  plausibility  measures,  the  two  dual  nonadditive  measures  of  DST, 
are  commonly  interpreted  as  lower  and  upper  probabilities,  respectively.  Although  DST 
is  now  mathematically  well  developed  [Shafer  1976,  Guan  and  Bell  1991-92]  and  has  a 
great  potential  applicability  in  many  areas  [Yager  et  al.  1994],  its  practical  utility  has  been 
hampered  by  the  lack  of  efficient  methods  for  constructing  belief  measures  (or  the  dual 
plausibility  measures)  in  the  context  of  specific  applications.  To  alleviate  this  weakness 
in  the  applicability  of  DST,  relevant  research  was  conducted  as  a  part  of  this  project. 

The  problem  of  constructing  belief  measures  was  recognized  as  a  special  case  of  the 
broader  problem  of  constructing  arbitrary  fuzzy  measures.  This  makes  sense  since 
imprecise  probabilities  can  be  represented  within  fuzzy  measure  theory  in  more  than  one 
way.  Six  distinct  strategies  for  constructing  fuzzy  measures  were  investigated: 
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•  Constructing  a  new  fuzzy  measure  from  a  given  fuzzy  measure  by  the  Sugeno  integral 
or  the  Choquet  integral  and  a  suitable  class  of  parameterized  functions  [4,14,15]; 

•  Constructing  new  fuzzy  measures  from  given  fuzzy  measures  by  suitable 
transformations  [12,13]; 

•  Constructing  fuzzy  measures  from  data  by  special  neural  networks  or  genetic 
algorithms  [11,23,39,40,42]; 

•  Constructing  various  types  of  fuzzy  measures  via  their  representations  in  terms  of  the 
usual  semantics  of  propositional  modal  logic  [3]; 

•  Constructing  fuzzy  measures  on  given  classes  of  sets  by  extensions  from  fuzzy 
measures  defined  on  smaller  classes  [1,26,37]; 

•  Constructing  fuzzy  measures  from  application  constraints  by  using  the  principle  of 
maximum  uncertainty  [5]. 

Technical  summary  of  these  broad  investigations,  only  some  of  which  were  directly 
connected  with  this  project,  is  presented  in  an  expository  paper  by  Klir,  Wang,  and 
Harmanec  [1997]. 

Among  these  broad  investigations  of  methods  for  constructing  fuzzy  measures,  two 
methods  resulting  from  these  investigations  are  explicitly  oriented  to  constructing  specific 
imprecise  probabilities.  One  of  them  is  a  powerful  method  for  constructing  belief 
measures  from  input-output  data  by  learning  via  a  special  neural  network  [40].  The  other 
one  is  a  method  for  constructing  Sugeno  measures  (Sec.  4.1)  via  a  special  genetic 
algorithm  [39].  As  is  well  known,  Sugeno  measures  capture  a  significant  subset  of  the 
dual  pairs  of  belief  and  plausibility  measures  of  DST,  but  they  are  considerably  easier  to 
represent  and  manipulate. 

The  first  method  is  based  on  a  representation  of  the  Choquet  integral  by  a  special 
nonlinear  neural  network  with  five  layers  [40].  The  Choquet  integral,  |  fd)i.  ,  of  a 
nonnegative  real-valued  function  f  on  X  with  respect  to  a  fuzzy  measure  \l  on  the  power 
set  of  X  is  defined  by  the  formula 


jfd|i  =  J|i(“F)da, 

0 

where  “F  =  {x  e  x|f(x)  >  a}  for  any  a€  [0,  1].  When  |J,  is  a  classical  measure  (i.e., 
additive),  the  Choquet  integral  coincides  with  the  Lebesque  integral.  When  p.  is  a  belief 
measure  (or  plausibility  measure),  the  Choquet  integral  is  a  coherent  lower  prevision  (or 
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upper  prevision,  respectively)  in  the  sense  of  the  theory  of  imprecise  probabilities  [1,  26]. 
Given  a  set  of  input-output  pairs,  each  consisting  of  a  tuple  (f(x)  |  x  e  X)  and  the 
associated  value  of  the  Choquet  integral,  the  neural  network  is  capable  of  solving  the 
inverse  problem,  the  problem  of  determining  the  fuzzy  measure  with  respect  to  which  the 
integration  takes  place. 

The  second  method  is  also  based  on  the  Choquet  integral,  but  it  involves  Sugeno  X  - 
measures,  which  capture  only  a  subset  of  belief  and  plausibility  measures.  It  solves  the 
inverse  problem  by  a  suitable  genetic  algorithm. 


6.2  Uncertainty  Measure 

Shortly  before  work  on  this  project  began,  a  new  measure  of  uncertainty  and  information 
in  DST  was  developed.  This  measure  was  shown,  contrary  to  all  its  competitors,  to  satisfy 
all  mathematical  properties  required  for  such  a  measure:  subadditivity,  additivity,  proper 
range,  and  appropriate  boundary  conditions  [Harmanec  and  Klir  1994].  For  each  finite 
frame  of  discernment  X  and  each  belief  function  defined  on  p(K),  the  measure  of 
uncertainty  is  a  function  AU  defined  by 

AU(Bel)  =  max]-XPxIog2p, 

L  xeX 

where  the  maximum  is  taken  over  all  probability  distributions  (p  Jx  e  x)  that  satisfy  for 
all  A  e  p(x)  the  constraints 

Bel(A)<Xp^. 

xeA 


Since  function  AU  is  formulated  as  the  solution  to  a  nonlinear  optimization  problem,  its 
utility  appeared  initially  rather  bleak.  Fortunately,  a  relatively  simple  and  fully  general 
algorithm  for  computing  the  function  was  discovered  and  its  correctness  proven  when 
working  on  this  project  [17].  The  existence  of  this  algorithm  made  the  measure  practical 
and  opened  a  way  for  developing  generalized,  DST-based  information  theory  [19,29,31, 
32]. 


The  algorithm  for  computing  the  uncertainty  measure  AU  is  formulated  as  follows: 
Input:  a  frame  of  discernment  X,  a  belief  function  Bel  on  X. 

Output:  AU(Bel),  [pxlxex  such  that  AU(Bel)  =  xexPx  logz  p,,Pi  >  0,^  xexPx  =  I 
and  Bel(A)  <  ^  ^^^^p^for  all  0  A  e  X. 


Step  1.  Find  a  non-empty  set  A  c  X,  such  as  p,^ 


Bel(A) 

|A| 


is  maximal.  If  there  are  more 
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such  sets  A,  take  one  with  maximal  cardinality. 

Bel(A) 

Step  2.  For  x  e  A,  put  — . 

Step  3.  For  each  B  c  X  -  A,  put  Bel(B)  =  Bel(B  u  A)  -  Bel(A). 

Step  4.  Put  X  =  X  -  A. 

If  X  ^*0  and  Bel(X)  >  0,  then  go  to  Step  1. 

Step  6.  If  Bel(X)  =  0  and  X  /  0,  then  put  px  =  0  for  all  x  e  X. 

Step  7.  Calculate  AU(Bel)  =  xexPx  Px- 

Stop. 

In  addition  to  this  general  algorithm,  a  special  and  very  efficient  algorithm  was  also 
developed  that  is  applicable  only  to  possibilistic  bodies  of  evidenee  [3,  17]. 


6.3  Uncertainty  Principles 

Although  the  utility  of  the  well-justified  measure  of  uncertainty  in  DST  is  as  broad  as  the 
utility  of  any  relevant  measuring  instrument,  its  role  is  particularly  pronounced  in  three 
fundamental  principles  for  dealing  with  uncertainty  and  the  associated  uncertainty-based 
information.  These  principles  are.-  principle  of  minimum  uncertainty,  principle  of 
maximum  uncertainty,  and  principle  of  uncertainty  invariance  [5].  The  first  two 
principles,  when  formulated  within  DST,  are  generalizations  of  the  well-known  principles 
of  minimum  and  maximum  entropy  of  classical,  probability-based  information  theory 
[Christensen  1985,  1986]. 

The  third  uncertainty  principle,  the  principle  of  uncertainty  invariance,  is  of  relatively 
recent  origin  [Klir  1990].  It  was  introduced  to  facilitate  meaningful  transformations 
between  the  various  uncertainty  theories.  According  to  this  principle,  the  amount  of 
uncertainty  (and  the  associated  uncertainty-based  information)  should  be  preserved  in 
each  transformation  of  uncertainty  from  one  mathematical  framework  to  another. 

The  uncertainty-invariance  principle  was  first  applied  to  probability-possibility 
transformations.  During  the  initial  investigation,  unfortunately,  no  well  justified  measure 
of  possibilistic  uncertainty  was  available.  Hence,  the  results  were  recently  revised,  as  part 
of  this  project,  in  terms  of  the  well-justified  measure  AU  (Sec.  6.2)  and,  in  addition, 
transformations  from  DST  to  possibility  theory  (possibilistic  approximations  of  belief 
measures)  were  analyzed  [3,18,30,32]. 

The  prineiple  of  uncertainty  invariance  is  applicable,  for  example,  for  obtaining 
meaningful  probabilistic  or  possibilistic  approximations  of  given  bodies  of  evidence  in 
DST,  or  for  obtaining  meaningful  approximations  of  fuzzy  sets  by  crisp  sets.  The 
motivation  to  apply  the  prineiple  to  formalizing  meaningful  transformations  between 
probabilistic  and  possibilistic  formalization  of  the  same  situation  has  arisen  not  only  from 
our  desire  to  comprehend  the  relationship  between  the  two  theories  of  uneertainty,  but 
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also  from  practical  problems.  Examples  of  these  problems  are:  constructing  a 
membership  grade  function  of  a  fuzzy  set  from  statistical  data,  combining  probabilistic 
and  possibilistic  information  in  expert  systems,  constructing  a  probability  measure  from  a 
given  possibility  measure  in  the  context  of  decision  making  or  systems  modeling,  or 
converting  given  probabilities  to  possibilities  to  reduce  computational  complexity. 


7.  OTHER  ISSUES  RELEVANT  TO  IMPRECISE  PROBABILITIES 


7.1  Fuzzy  Measure  Theory 

In  addition  to  the  investigations  regarding  the  various  strategies  for  constructing  fuzzy 
measures,  in  particular  belief  measures  and  Sugeno  measures  (Sec.  6.1),  some  important 
theoretical  results  were  obtained  in  fuzzy  measure  theory.  These  results,  which  are 
relevant  to  imprecise  probabilities  to  various  degrees,  involve,  for  example,  some 
theoretical  issues  regarding  the  natural  extension  of  lower  probabilities  [26,37]  or  the 
roles  of  various  nonlinear  integrals  (Sugeno  integrals,  Choquet  integrals,  pan-integrals, , 
etc.)  in  dealing  with  imprecise  probabilities  [9,14,15,27,28].  Since  results  in  this  category 
are  overly  technical  and  involve  many  distinct  issues,  we  do  not  attempt  to  summarize 
them  here  to  avoid  a  presentation  that  would  inevitably  be  prohibitively  long.  It  seems 
more  prudent  in  this  case  to  refer  to  the  above  listed  papers  for  full  coverage  of  the  issues 
involved. 


7.2  Constrained  Fuzzy  Arithmetic 

In  the  process  of  developing  Bayesian  inference  with  fuzzy  probabilities  (Sec.  5.2),  it 
became  evident  that  standard  fuzzy  arithmetie  [Kaufman  and  Gupta  1985]  is  not 
adequate.  For  any  finite  set  of  elementary  events,  probabilities  of  the  events  are  required 
to  add  to  1.  When  the  numerical  probabilities  are  approximated  by  fuzzy  probabilities, 
the  sum  of  these  fuzzy  probabilities  is  not  1  by  the  rules  of  standard  fuzzy  arithmetic.  In 
order  to  deal  with  fuzzy  probabilities  in  a  meaningful  way,  the  requirement  that  they 
always  add  to  exactly  1,  regardless  how  imprecise  they  are,  must  be  added  to  fuzzy 
arithmetic  as  a  requisite  constraint. 

After  reeognizing  the  requisite  constraint  imposed  on  arithmetic  operations  with  fuzzy 
probabilities  by  axioms  of  probability  theory,  one  realizes  that  there  are  also  other 
requisite  constraints  that  must  be  considered  when  using  fuzzy  arithmetic.  Perhaps  the 
most  common  of  them  is  the  requisite  equality  constraint;  given  any  arithmetic  expression 
with  fuzzy  intervals,  those  fuzzy  intervals  represented  by  the  same  symbol  (i.e., 
expressing  approximately  the  value  of  the  same  linguistic  variable)  must  be  treated  as 
numerically  equal. 
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Any  constraint  associated  with  fuzzy  intervals  on  which  we  operate  contains  some 
information.  When  constraints  are  ignored,  as  in  standard  fuzzy  arithmetic,  the  obtained 
results  are  invariably  information  deficient,  which  means  that  they  are  overly  imprecise. 
This  feature  of  standard  fuzzy  arithmetic  to  produce  results  with  inflated  imprecision 
made  some  practitioners  abandon  fuzzy  arithmetic  as  impractical. 

To  guarantee  that  imprecision  be  not  inflated  when  applying  arithmetic  operations  to 
fuzzy  intervals,  all  constraints  regarding  the  fuzzy  intervals  involved  must  be  taken  into 
account.  This  may  include  indigenous  constraints  (such  as  the  equality  constraints)  as 
well  as  various  extraneous  constraints  (such  as  the  constraint  imposed  by  probability 
theory  when  dealing  with  fuzzy  probabilities). 

Incorporating  constraints  into  fuzzy  arithmetic  results,  unfortunately,  in  greater 
computational  complexity.  This  is  due  to  the  fact  that  arithmetic  expressions  subjected  to 
constraints  must  be  evaluated  globally  and  not  by  appropriately  sequenced  local 
arithmetic  operations.  Moreover,  the  global  evaluation  can  often  be  accomplished  only  by 
solving  specific  optimization  problems  [43]. 

Constrained  fuzzy  arithmetic  has  not  been  sufficiently  developed  as  yet,  even  though 
basic  issues  pertaining  to  it  have  already  been  identified  [34,  35,  43].  Some  challenging 
mathematical  and  computational  problems  will  have  to  be  solved  to  make  it  fully 
operational  and  useful  within  the  broader  area  of  soft  computing  [Bonissone  1997,  Wang 
and  Tan  1997].  Fortunately,  a  lot  can  be  learned  by  examining  the  many  relevant  results 
that  have  already  been  obtained  in  the  area  of  interval  computation  [Neumaier  1990, 
Kearfott  and  Kreinovich  1996]. 


7.3  Mathematics  of  Finite  Resoiution 

Imprecise  probabilities  are  used  as  rough  estimates  whenever  we  do  not  have  the 
capability  to  obtain  precise  probabilities.  There  are  various  ways  in  which  imprecise 
probabilities  can  be  expressed:  intervals  of  real  numbers,  fuzzy  intervals,  convex  sets  of 
probability  distributions,  rough  sets,  etc.  This  raises  an  interesting  question:  Is  there  some 
underlying  theory,  by  which  these  seemingly  dissimilar  characterizations  of  imprecision 
of  probabilities  (or  some  other  quantities  of  interest)  could  be  treated  in  a  unified  way? 
This  issue  was  investigated  in  the  context  of  this  project.  It  was  found  that  such  a  theory, 
for  which  the  name  theory  of  finite  resolution  was  coined,  is  possible  and,  when 
adequately  developed,  would  enhance  our  understanding  of  the  phenomena  of 
imprecision  and  uncertainty. 

In  this  study  [20],  the  concept  of  a  finite  resolution  was  defined  as  a  process  that  returns 
increasingly  better  approximations  to  an  object  defined  by  a  function  or  a  set  of  functions 
from  some  appropriate  function  space.  The  process  is  formalized  as  a  sequence  of 
operators  parameterized  by  a  resolution  parameter,  which  quantifies  our  ability  to  discern 
details.  A  few  basic  ideas  pertaining  to  the  theory  were  developed  and  some  problems  to 
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which  it  could  be  applied  were  identified  [21],  However,  this  work  is  only  loosely 
connected  with  the  objective  of  this  project  and,  therefore,  it  was  not  pursued  further 
within  the  project.  It  is  expected  that  further  research  on  the  theory  will  be  resumed 
under  a  more  fitting  project  in  the  future. 


7.4  Critical  Examination  of  Cox’s  Proof 

The  claim  that  the  only  correct  way  to  deal  with  uncertainty  is  to  use  the  rules  of 
probability  theory  has  been  justified  by  referring  to  a  sequence  of  intuitive  and  formal 
arguments  presented  by  Cox  [1946].  These  arguments  are  usually  referred  to  in  the 
literature  as  the  Cox’s  proof,  even  though  the  arguments  are  not  presented  as  a  proof  of 
mathematically  stated  theorem.  The  aim  of  the  Cox’s  proof  is  to  show  that  the  product 
rule  of  classical  Bayesian  inference  can  be  derived  mathematically  as  the  only  possible 
operation  by  properly  formalizing  our  intuitions  about  rational  reasoning.  Cox  argues 
that  these  intuitions  can  be  expressed  in  terms  of  specific  functional  equations  whose 
solution  is  the  product  rule.  However,  the  proof  is  based  on  some  hidden  assumptions, 
which  are  not  explicitly  stated  by  Cox.  Although  these  assumptions  have  already  been 
used  as  a  basis  for  criticizing  the  proof  from  various  points  of  view,  none  of  these 
criticisms  is  fully  comprehensive.  As  a  result,  the  ongoing  debate  regarding  this  very 
fundamental  issue,  as  summarized  in  [41],  became  considerably  clouded.  To  clarify  the 
meaning  of  the  Cox’s  theorem,  its  comprehensive  examination  was  undertaken  in  the 
context  of  this  project  [8,41].  Two  important  results  were  obtained  by  this  examination: 
(i)  the  Cox’s  proof  does  not  support  the  alleged  claims  that  the  rules  of  probability  are  the 
only  coherent  rules  to  deal  with  uncertainty;  and  (ii)  there  exist  meaningful  Bayesian-like 
inference  rules  for  dealing  with  probabilities  of  fuzzy  events,  which  are  different  from  the 
classical  Bayesian  inference. 


8.  CONCLUSIONS 

The  purpose  of  this  section  is  twofold,  (i)  To  identify  the  principal  results  emerging  from 
the  work  on  this  project  and  to  assess  their  significance;  and  (ii)  To  identify  new  problem 
areas,  which  were  opened  by  some  of  the  results,  and  assess  their  importance  for  future 
research  projects. 

The  following  are  principal  results,  conveniently  clustered  according  to  similarities  in 
their  meaning: 

1.  Several  efficient  methods  were  developed  for  Bayesian  inference  with  interval-valued 
or  fuzzy  probabilities,  which  are  well-grounded  in  theory  [10,24,25,33,38].  The 
significance  of  these  methods  is  that  they  produce  posterior  probabilities  with  exact 
lower  and  upper  bounds  or  with  exact  membership  functions  defining  the  respective 
fuzzy  intervals.  No  comparable  method,  among  those  described  in  the  literature,  is 
capable  of  achieving  this;  all  these  methods  can  produce  only  various  estimates  of  the 
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lower  and  upper  bounds  or  membership  functions  of  posterior  probabilities.  The 
methods  developed  in  this  project  for  Bayesian  inference  with  imprecise  probabilities 
are  applicable  to  any  problem  area  where  classical  Bayesian  inference  is  applicable  in 
principle,  but  its  actual  use  is  compromised  since  precise  prior  probabilities  or 
likelihoods  cannot  be  obtained.  This  utility  of  the  methods  is  illustrated  in  [33]  by  two 
problem  areas:  the  problem  area  of  military  unit  identification  and  the  problem  area  of 
statistical  decision  making.  The  first  problem  area  was  chosen  as  a  response  to  the 
following  statement  in  Rome  Laboratory  Final  Technical  Report  No.  RL-TR-95-99  : 
“(The  Bayes’  theorem)  does  not  account  for  issues,  such  as  errors  in  observation 
vectors  or  uncertainties  in  prior  assumptions  utilized  by  the  probability  models.  So  the 
problem  is  how  to  identify  enemy  military  units  in  battlefield  through  the  observed 
imprecise  information  and  imprecise  prior  knowledge  of  enemy  military  doctrine.” 
The  second  problem  area,  statistical  decision  making,  was  chosen  because  of  its  broad 
applicability. 

2.  Computationally  efficient  methods  were  developed  for  determining  imprecise 
probabilities,  either  from  human  judgment  or  from  given  data.  For  eliciting  imprecise 
probabilities  from  domain  experts,  a  counterpart  of  Saaty’s  method  of  hierarchical 
analysis  [Saaty  1980]  was  developed,  which  is  based  on  differences  rather  than  ratios 
in  comparing  relevant  alternatives  and  criteria  [24,  33].  The  significance  of  this 
method  is  that  its  computational  complexity  is  considerably  lower  in  comparison  with 
Saaty’s  original  method  and,  hence,  its  practical  utility  is  significantly  higher.  For 
constructing  imprecise  probabilities  (and  also  some  more  general  fuzzy  measures) 
from  data,  several  methods  based  on  special  neural  networks  or  genetic  algorithms 
were  developed  [11,39,40,42],  The  significance  of  these  methods  is  their 
computational  efficiency,  as  well  as  their  capability  to  obtain  good  approximate 
solutions  when  no  exact  solutions  exist  and  ,  hence,  when  analytic  methods  are  not 
applicable. 

3.  An  efficient  algorithm  was  developed  for  computing  a  previously  discovered  well- 
justified  measure  of  uncertainty  and  uncertainty-based  information  in  Dempster- 
Shafer  theory  (DST)  [17],  Such  an  algorithm  was  essential  for  making  the  measure 
practicable  and,  hence,  to  make  the  prospective  development  of  DST-based 
information  theory  feasible.  The  algorithm  and  the  principle  of  uncertainty  invariance 
[5]  were  also  utilized  to  formulate  operational  procedures  for  sound  transformations 
between  possibility  theory  and  probability  theory,  as  well  as  procedures  for 
transformations  from  DST  to  possibility  theory  or  probability  theory  [3,18,30,32].  In 
addition,  a  special  function  was  established  for  the  first  time  on  sound  axiomatic 
grounds  for  measuring  nonspecificity  of  convex  fuzzy  subsets  of  the  n-dimensional 
Euclidean  space  [6].  This  function  is  applicable,  for  example,  for  measuring 
information  obtained  by  any  imprecise  measurements  regarding  the  location  of  an 
object  in  space. 

4.  A  careful  mathematical  analysis  was  made  of  hidden  assumptions  in  the  well  known 
proof  by  Cox  [1946],  whose  aim  was  to  justify  the  classical  Bayesian  inference  as  the 


sole  agent  for  reasoning  under  uncertainty  [2,41],  This  analysis  has  important 
implications  since  it  shows  that  the  proof  is  invalid  and  does  not  block  alternative 
rules  of  inference,  distinct  from  the  classical  Bayesian  rules,  for  Bayesian-like 
inference  based  on  fuzzy  events.  Although  some  meaningful  Bayesian-like  inference 
rules  have  already  been  derived  and  are  presented  in  [41],  further  research  is  needed 
to  develop  this  new  area. 

5.  As  a  byproduct  of  research  on  imprecise  probabilities  within  the  broad  framework  of 
fuzzy  measure  theory,  numerous  special  mathematical  results  were  obtained  by  which 
fuzzy  measure  theory  has  significantly  been  enhanced  [14,15,16,27,28]. 

6.  Several  computer  programs  were  developed  to  support  the  methodology  developed  in 
this  project  for  dealing  with  imprecise  probabilities.  Some  preliminary  steps  were  also 
taken  to  develop  a  shell  program,  equipped  with  a  user-friendly  interface,  in  which 
these  programs  and  other  relevant  programs  will  be  embedded. 

The  work  in  this  project  produced  not  only  specific  results,  as  presented  in  publications 

listed  in  Sec.  10,  but  it  also  opened  new  avenues  for  future  research.  Among  them,  the 

following  seems  to  be  the  most  significant  new  areas  of  research  that  emerged  from  the 

work  on  this  project: 

1.  The  algorithm  for  efficient  computation  of  uncertainty  in  DST  [17]  makes  it  feasible 
not  only  to  operationalize  the  principles  of  maximum  and  minimum  uncertainty  in 
DST  [5],  but  also  to  develop  the  calculus  of  generalized  DST-based  information 
theory. 

2.  Constrained  fuzzy  arithmetic  [22,34,35],  which  emerged  from  research  on  imprecise 
probabilities  in  this  project,  is  essential  to  virtually  all  applications  of  linguistic 
variables  that  involve  arithmetic  operations.  Some  results  were  already  obtained  in 
this  new  area  [34,35],  but  much  more  remains  to  be  done.  In  particular,  relevant 
optimization  problems  needed  to  facilitate  fuzzy  arithmetic  under  recognized  typed  of 
constraints  (as  emerging  from  various  applications)  must  be  formulated  and 
methodologically  developed  for  this  purpose. 

3.  While  basic  ideas  of  the  broad  theory  of  finite  resolution  were  formulated  in  the 
context  of  this  project  [20]  and  shown  useful  [21],  extensive  effort  will  be  needed  to 
fully  develop  it  and  explore  its  promising  applications  to  some  basic  problems  of 
measurement  and  computation. 

4.  Alternative  rules  for  Bayesian-like  inference  based  on  fuzzy  events,  which  emerged 
from  a  critical  analysis  of  the  Cox’s  proof  [2,41],  need  to  be  further  studied  and  their 
applications  examined.  In  particular,  counterparts  of  the  probability  calculus  need  to 
be  developed  for  the  alternative  rules  of  inference. 
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APPENDIX:  Computer  Software 


The  purpose  of  this  Appendix  is  to  describe  basic  characteristics  of  computer  programs 
implementing  the  various  algorithms  developed  in  this  project.  For  each  algorithm,  these 
characteristics  are:  the  problem  solved  by  the  algorithm,  requested  input,  produced 
output,  relevant  restrictions,  basic  steps  of  the  algorithm,  and  references  to  publications 
where  the  algorithm  is  described  in  detail  and  supported  by  a  relevant  proof  of  its  validity. 
The  use  of  each  program  is  also  illustrated  by  a  simple  example. 

A.  Bayesian  Inference  with  Imprecise  Probabilities 

In  this  section,  four  programs  are  described,  each  implementing  a  special  algorithm  for 
Bayesian  inference  with  imprecise  probabilities.  These  programs,  referred  to  as  Bayesian 
1  through  Bayesian  4,  differ  from  one  another  in  the  type  of  prior  probabilities  (interval¬ 
valued  or  fuzzy)  and  in  the  type  of  likelihoods  (precise,  interval-valued,  fuzzy),  which 
they  accept  as  inputs,  as  well  as  in  the  type  of  the  calculated  posterior  probabilities. 

A.1  Bayesian  1 :  Precise  Likelihoods  and  Interval-Valued  Prior 
Probabilities,  Interval-valued  Posterior  Probabilities  [33,  38] 

Problem 

Given  interval-valued  prior  probabilities  and  precise  likelihoods,  calculate  interval-valued 
posterior  probabilities. 

Inputs 

•  finite  universal  set:  X  =  {xi,  X2, . . .,  Xg) 

•  precise  likelihoods  on  X:  ai,  ...,as 

•  reasonable  prior  probability  intervals  on  X:  [h,  ui] ,  . .  .,[ls,  Ug] 

Output 

•  posterior  probability  intervals  on  X:  [IpU^  ],. . . ,[l',u'] 

Restrictions 

•  The  sum  of  the  lower  prior  probabilities  h,  . .  .,lg  must  be  less  than  or  equal  to  one  and 
the  sum  of  the  prior  upper  probabilities  ui,  ...,Us  must  be  greater  than  or  equal  to  1. 

Basic  Steps 

Step  1:  Based  on  the  given  reasonable  TPI,  compute  the  corresponding  feasible 
TPI<Fi,  ...,Fg>. 

Step  2:.  Using  the  precise  likelihoods  ai, . .  .,ag  and  the  feasible  TPI  <Fi, . . .,  Fg)  of  prior 
probabilities,  calculate  the  posterior  TPI(f,'  ,. . . , F/)  . 
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Example  [38] 

1.  Define  a  universal  set  by  entering  its  elements  xi,  X2,  X3,  X4. 

2.  Specify  the  following  likelihood  assignment: 


ai 

Xi 

0.25 

X2 

0.20 

X3 

0.30 

X4 

0.10 

3.  Specify  the  following  prior  probability  interval  assignment: 


li 

Ui 

Xl 

0.20 

0.25 

X2 

0.10 

0.50 

X3 

0.40 

0.45 

X4 

0.10 

0.15 

4.  Execute  Bayesian  1 .  The  resulting  output  specifying  the  posterior  TPI  is: 


v 

Ui' 

Xl 

0.2041 

0.2632 

X2 

0.1237 

0.2500 

X3 

0.4  949 

0.5625 

X4 

0.0404 

0.0638 

A.2  Bayesian  2:  Prior  Probabilities,  Likeiihoods,  and  Posterior 
Probabilities  are  Intervai-Valued  [33,  38] 

Probiem 

Given  interval-valued  prior  probabilities  and  likelihoods,  calculate  interval-valued 
posterior  probabilities. 

Inputs 

•  finite  universal  set :  X  =  {xi,  X2,  Xg}. 

•  likelihood  intervals  :  [ai,  bi] ,  ...,[as,  bg] 

•  reasonable  prior  probability  intervals:  [li,  ui] ,  . .  .,[ls,  Ug] 
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Output 

•  posterior  probability  intervals:  [1/,  uj'  ] ,  Us'] 


Restrictions 

•  The  sum  of  the  lower  probabilities  must  be  less  than  or  equal  to  one  and  the  sum  of 
the  upper  probabilities  must  be  greater  than  or  equal  to  1. 

Basic  Steps 

Step  1:  For  the  given  TPI  of  reasonable  prior  probabilities,  compute  the  corresponding 
feasible  TPI  <Fi,  . . .,Fs>. 

Step  2:  Using  (Fi,  . . .,Fs)  and  the  given  likelihood  intervals  (Gi,  . . .,G s>,  calculate  the 
posterior  TPI  (F/,  . .  .,Fs'>. 

Exampie  [25] 

1.  Define  a  universal  set  by  entering  its  elements  X],  X2,  X3,  and  X4. 

2.  Make  the  following  likelihood  interval  assignment; 


ai  bi 

xi  0.20  0.30 

X2  0.10  0.25 

X3  0.30  0.50 

X4  0.10  0.20 


3.  Make  the  following  prior  probability  interval  assignment: 


li 

Ui 

Xi 

0.20 

0.25 

X2 

0.10 

0.50 

X3 

0.40 

0.45 

X4 

0.10 

0.15 

1.  Execute  Bayesian  2.  The  resulting  output  specifying  the  posterior  TPI  is: 


li' 

Ui' 

Xl 

0.1151 

0.3261 

X2 

0.0435 

0.3061 

X3 

0.4324 

0.7500 

X4 

0.0278 

0.1395 
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A.3  Bayesian  3:  Prior  Probabilities  are  Trapezoidal  Fuzzy  Numbers, 
Likelihoods  are  Precise  Numbers,  and  Posterior  Probabilities  are 
Trapezoidal  Fuzzy  numbers  [25,  33] 


Problem 

Given  fuzzy  prior  probabilities  and  precise  likelihoods,  calculate  fuzzy  posterior 
probabilities. 

Inputs 

•  finite  universal  set:  X  =  { x  i ,  X2, . . . ,  Xs } . 

•  precise  likelihoods;  ai,  ...,as 

•  reasonable  prior  TFP  of  trapezoidal  type:  «ai,  bj,  c,,  d,  >, . .  .,<as,bs,  Cs,  ds» 

Output 

•  feasible  posterior  TFP:  «a,',  W,  c,',  d/), . bs',  c/,  d/» 

Restrictions 

•  for  all  prior  probabilities,  a;  <  bi  <  Ci  <  di, 

•  for  all  a-cuts  of  each  prior  probability,  the  sum  of  the  lower  bounds  must  be  <  1  and 
the  sum  of  the  upper  bounds  must  be  >  1. 

Basic  Steps 

Step  1:  Calculate  the  a-cuts  of  the  given  fuzzy  prior  probabilities  for  all  a  e  (0,1]. 

Step  2:  Calculate  the  TPI  (“Fi, . . .,  “Fs)  of  the  feasible  a-cuts  of  the  prior  fuzzy 
probabilities. 

Step  3:  Using  the  given  precise  likelihoods  <ai,  ...,as>  and  the  feasible  a-cuts 
(“Fi,  ...,“Fs>,  calculate  the  posterior  a-cuts  (“Fi',  ...,“Fs'). 

Step  4:  Calculate  trapezoidal  approximations  of  the  posterior  fuzzy  probabilities 
represented  by  the  a-cuts  obtained  in  Step  3. 

Example  [25,  33] 

1.  Define  a  universal  set  by  entering  its  elements  Xi,  X2,  X3,  and  X4. 

2.  Make  the  following  likelihood  assignment: 


ai 

Xl 

0.10 

X2 

0.20 

X3 

0.20 

X4 

0.30 
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3.  Make  the  following  probability  assignment; 


s-i  bi  Cj  di 


Xl 

0.00 

0.10 

0.10 

0.20 

X2 

0.10 

0.20 

0.20 

0.25 

X3 

0.20 

0.30 

0.30 

0.40 

X4 

0.30 

0.40 

0.40 

0.60 

4.  Execute  Bayesian  3.  The  resulting  output  is: 


Si' 

bi' 

Ci' 

di' 

Xl 

0.0000 

0.0435 

0.0435 

0.0942 

X2 

0.0744 

0.1738 

0.1738 

0.2369 

X3 

0.1512 

0.2606 

0.2606 

0.3787 

X4 

0.4008 

0.5215 

0.5215 

0.7237 

A.4  Bayesian  4:  Prior  Probabilities,  Likelihoods,  and  Posterior 
Probabilities  are  Trapezoidal  Fuzzy  Numbers  [25,  33] 


Problem 

Given  fuzzy  prior  probabilities  and  fuzzy  likelihoods,  calculate  fuzzy  posterior 
probabilities. 

Inputs 

•  finite  universal  set :  X  =  {xi,  X2,  . . Xs} 

•  fuzzy  likelihoods  of  trapezoidal  type:  «a,,b,,Ci,di>, <a3,b3,c,,d,» 

•  reasonable  TFP  of  trapezoidal  type:  ((ai,  bi,  ci,  di ), {as,bs,  Cg,  dg)) 

Output 

•  feasible  posterior  TFP  of  trapezoidal  type:  ((a/,  b/,  ci ,  di  ), . .  .,(ag ,  bg ,  Cg ,  dg )) 

Restrictions 

•  for  all  trapezoidal  likelihoods  a ;  <  b ;  <  c  j  <  d ; 

•  for  all  trapezoidal  prior  probabilities,  ai  <  bi  <  Ci  <  di. 

•  for  all  a-cuts  of  each  prior  probability,  the  sum  of  the  lower  bounds  must  be  <  1  and 
the  sum  of  the  upper  bounds  must  be  >  1. 


Basic  Steps 
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Step  1:  Calculate  the  a-cuts  of  the  given  fuzzy  prior  probabilities  for  a  e  (0,1], 

Step  2:  Calculate  the  TPI  {“Fi,  ...,“Fs>  of  feasible  a-cuts  of  the  prior  fuzzy  probabilities. 
Step  3:  Calculate  the  a-cuts  <“Gi,  . . . G  s>  of  fuzzy  likelihood  Gi,  . .  .,Gs. 

Step  4:  Using  the  a-cuts  (“Gi, Gs)  and  the  feasible  a-cuts  (“Fi,  ...,“Fs>,  calculate  the 
a-cuts  (“Fi',“F2',.  .  .,“Fs'>  of  posterior  probabilities. 

Step  5:  Calculate  trapezoidal  approximations  of  the  posterior  fuzzy  probabilities 
represented  by  the  a-cuts  obtained  in  Step  4. 

Example  [25] 

1.  Define  a  universal  set  by  entering  elements  xi,  X2,  and  X3. 

2.  Make  the  following  likelihood  assignment: 


dj 

bi 

Ci 

di 

Xi 

0.10 

0.25 

0.25 

0.30 

X2 

0.20 

0.30 

0.30 

0.50 

X3 

0.30 

0.40 

0.40 

0.60 

3.  Make  the  following  probability  assignment: 


Si 

bi 

Ci 

di 

Xl 

0.00 

0.00 

0.20 

0.40 

X2 

0.10 

0.30 

0.40 

0.50 

X3 

0.40 

0.60 

0.80 

0.90 

4.  Execute  Bayesian  4.  The  resulting  output  specifying  the  posterior  TFP  is: 


ai' 

bi' 

Ci' 

di' 

Xl 

0.0000 

0.0000 

0.0704 

0.4366 

X2 

0.0119 

0.2432 

0.3334 

0.6577 

X3 

0.2918 

0.6667 

0.7568 

0.9881 

B.  Uncertainty  in  Dempster-Shafer  Theory  [17] 

Probiem 

Given  a  body  of  evidence  represented  by  a  belief  function  Bel  of  DST,  calculate  the 
associated  overall  uncertainty. 


Inputs 

•  finite  universal  set  (frame  of  discernment) :  X 
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Bel(A)  for  each  A  in  the  power  set  of  X 


Output 

•  AU(Bel) 

Restrictions 

•  Set  function  Bel  must  satisfy  all  properties  of  belief  measures,  as  defined  in  DST . 

Basic  Steps 

See  Sec.  6.2,  pp.  22-23. 


Example  [17] 

1 .  Define  a  universal  set  by  entering  its  elements  a,  b,  and  c. 

2.  Generate  the  power  set. 

3.  Specify  the  following  belief  measure  assignment  as  input; 

A  Bel(A) 

{a}  0.26 

{b}  0.26 

{c}  0.26 

{d}  0.00 

{a,b}  0.59 

{a,c}  0.53 

{a,d}  0.27 

{b,c}  0.53 

{b,d}  0.27 

{c,d}  0.27 

{a,b,c}  0.87 
{a,b,d}  0.61 
{a,c,d}  0.55 
{b,c,d}  0.55 
{a,b,c,d}  1.00 


4.  Execute  Compute  AU.  The  resulting  output  is:  1.93598. 


41 


C.  Natural  Extension  of  Lower  Probabilities  [37] 

Problem 

Given  a  finite  universal  set  X,  a  nonnegative  function  on  X,  and  a  set  function 
|I=  P(X)  ->  [0,l]  such  that  |J,(0)  =  0  and  |x(X)  =  1  (a  lower  probability),  calculate  the 
natural  extension,  E(f),  of  f  with  respect  to  p.. 

Inputs 

•  finite  universal  set:  X  =  {xi,  X2, ...,  Xs} 

•  nonegative  function  f  on  X 

•  set  function  p  on  the  power  set  of  X 

Output 

•  value  E  of  the  natural  extension  of  f  with  respect  to  p 

•  assignment  of  values  X,j  to  all  sets  Ej  of  the  power  set  [37] 

Restrictions 

•  f(x)  e  [0, 1  ]  for  all  xe  X 

•  p(A)  e  [0, 1]  for  all  A  in  the  power  set 

•  p(0)  =  0  and  p(X)  =  1 

Basic  Steps 

See  Ref.  [37]. 

Example  [37] 

1.  Define  a  universal  set  by  entering  its  elements  named  xi,  X2,  and  X3. 

2.  Define  the  following  function; 

X  f(Xi) 

xi  0.50 

X2  0.10 

X3  0.70 

3 .  Generate  the  power  set  of  [  x  1 ,  X2,  X3 } . 

4.  Define  the  following  set  function: 
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P(X) 

X2  Xi  U(Ei) 

0  0  0  0.00 

0  0  1  0.20 

0  1  0  0.30 

0  1  1  0.40 

1  0  0  0.10 

1  0  1  0.20 

1  1  0  0.60 

1  1  1  1.00 


5.  Execute  Natural  Extension.  The  resulting  output  is: 

•  E  =  0.71 

•  X3  X2  Xi _ X.i 

0  0  0  0.00 

0  0  1  0.00 

0  1  0  0.30 

0  1  1  0.00 

1  0  0  0.00 

1  0  1  0.20 

1  1  0  0.00 

1  1  1  0.50 


D.  Construction  of  X-Fuzzy  Measures  From  Given  Input-Output  Data  [11,  39] 
Problem 

Given  a  set  of  input-output  pairs,  where  each  input  is  an  n-tuple  of  real  numbers  in  [0,  1] 
that  represent  values  of  a  function  f  defined  on  a  set  X  with  n  elements  and  each  output  is 
a  real  number  in  [0,  1  ]  that  is  viewed  either  as  the  Choquet  integral  or  the  fuzzy  integral 
of  f  with  respect  to  an  unknown  X-fuzzy  measure  p,  determine  p  that  conforms  to  the 
given  data  as  closely  as  possible  in  terms  of  the  root-mean-square  error. 


Inputs 

•  finite  universal  set:  X  =  {xi,  X2, . . .,  Xs} 

•  input-output  data 

•  Choquet  integral  or  fuzzy  integral 
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Outputs 

•  value  of  parameter  X,  by  which  a  ?i-fuzzy  measure  is  characterized 

•  values  of  measure  p  for  all  singleton  sets 

•  root-mean-square  error 

•  number  of  iterations  employed 

Restrictions 

•  All  entries  in  input-output  data  must  be  real  numbers  in  [0,  1]. 

Basic  Steps 

A  special  genetic  algorithm  is  employed;  see  Ref.  [39]  for  details. 

Example 

Several  examples  are  described  in  Ref.  [39].  Due  to  the  random  search  employed  in  the 
genetic  algorithm,  these  examples  are  not  exactly  reproducible.  Hence,  we  do  not  present 
them  here. 
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